Image display apparatus and image display method

ABSTRACT

A number-of-pixel acquisition unit acquires the number of pixels of the image data stored in a storage device. An aspect ratio acquisition unit acquires the aspect ratio of the image data. A display control unit displays a list of a plurality of thumbnail images of the image data on a display device. Herein, the display control unit displays a thumbnail of image data having more pixels as a larger thumbnail than that of image data having less pixels, while keeping the aspect ratio of the image data.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus and animage processing method.

2. Description of the Related Art

With the wide spread of image-taking apparatuses, such as digitalcameras and cellular phones with a camera, in recent years, users canreadily acquire digital image data. Even with the camera function of acellular phone with a camera, the digital image data of about severalmillion pixels can be taken; and with a compact digital camera, thedigital image data of more than ten million pixels can be taken.Further, high-end single-lens reflex cameras, which have been usedmainly in professional applications, are being widely used by ordinaryusers.

Along with the spread of such image-taking apparatuses, there areincreasing occasions when digital image data are taken. For example,some of the users who have established their blogs take several ten orseveral hundred shots of digital image data per day. As understood fromwhat has been described so far, the number of the digital image data tobe taken is explosively increasing. On the other hand, the recordingapparatuses with a large capacity in which a large amount of suchdigital image data can be recorded are also being widely used.

When a large amount of image data are stored in a recording apparatus,there can be the case where it takes a long time for a user to searchfor the desired image. Also, there can be the case where a user forgetswhat type of images have been recorded in the recording apparatus. Asstated above, when the number of the image data stored in a recordingapparatus is large, there is sometimes the case where it is difficult tomanage the image data.

SUMMARY OF THE INVENTION

The present invention has been made in view of these situations, and apurpose of the invention is to provide a technique in which a largeamount of digital image data can be readily managed.

In order to solve the aforementioned problem, an embodiment of thepresent invention is an image processing apparatus. The apparatuscomprises: a first display control unit configured to display a list ofthumbnail images of image data on a display device; a directionreceiving unit configured to acquire a user's selection for the imagedata; and a second display control unit configured to display a list ofthumbnail images of one or more of image data that have been selected bythe user, in a display region different from the display region on thedisplay device where the thumbnail images to be displayed by the firstdisplay control unit are displayed, and simultaneously with the displayof the thumbnail images to be displayed by the first display controlunit. Herein, the first display control unit acquires the user'sselection from the direction receiving unit and displays, on the displaydevice, the thumbnail images of the image data that have been selectedby the user and the thumbnail images of the image date that have notbeen selected by the user, in different display modes.

Another embodiment of the present invention is an image processingmethod executable by a processor. The method comprises: displaying alist of thumbnail images of image data on a display device; acquiring auser's selection for the image data; displaying a list of thumbnailimages of one or more of image data that have been selected by the user,in a display region different from the display region where thethumbnail images to be displayed are displayed on the display device,and simultaneously with the display of the thumbnail images of the imagedata; and displaying, on the display device, the thumbnail images of theimage data that have been selected by the user and the thumbnail imagesof the image data that have not been selected by the user, in differentdisplay modes.

Still another embodiment of the present invention is an image displayapparatus. The apparatus comprises: a number-of-pixel acquisition unitconfigured to acquire the number of pixels of image data stored in astorage device; an aspect ratio acquisition unit configured to acquirethe aspect ratio of the image data; and a display control unitconfigured to display a list a plurality of thumbnail images of theimage data on a display device. Herein, the display control unitdisplays a thumbnail image of image data having more pixels as a largerthumbnail image than that of image data having less pixels, whilekeeping the aspect ratio of the image data.

Still another embodiment of the present invention is an image displaymethod executable by a processor. The method comprises: acquiring thenumber of pixels of image data stored in a storage device; acquiring theaspect ratio of the image data; and displaying, on a display device, alist of a plurality of thumbnail images of the image data having morepixels as a larger thumbnail image than that of the image data havingless pixels, while keeping the aspect ratio of the image data.

Still another embodiment of the present invention is also an imagedisplay apparatus. The apparatus comprises: a number-of-pixelacquisition unit configured to acquire the number of pixels of imagedata stored in a storage device; and an image display control unitconfigured to generate a plurality of thumbnail images in a manner inwhich an enlargement/reduction ratio used when a thumbnail image ofimage data with more pixels is to be generated is made smaller than thatused when the thumbnail image of the image data with less pixels is tobe generated, and to display a list of the generated thumbnail images ona display device.

Still another embodiment of the present invention is also an imagedisplay apparatus. The apparatus comprises: an image classification unitconfigured to classify image data to which attribute information isadded into groups based on the attribute information so as to generateone or more image groups; and a display control unit configured todisplay, on a display device, a list of a plurality of thumbnail imagesof the image data included in each of the image groups generated by theimage classification unit, in a manner in which the plurality ofthumbnail images are piled up for every image group.

Still another embodiment of the present invention is also an imagedisplay apparatus. The apparatus comprises: a display of a list controlunit configured to display, on a display device, a list of a pluralityof thumbnails of the image data to which attribute information is added,the image data being stored in a storage device; a direction receivingunit configured to acquire a user selection for the image data; and adisplay control unit configured to acquire, from the storage device, thepredetermined number of sheets of image data including attributeinformation associated with the attribute information included in aselected image, to arrange the predetermined number of sheets of imagedata within a virtual three-dimensional space, and to display, on thedisplay device, a walk-through video of the predetermined number ofsheets of image data.

Still another embodiment of the present invention is also an imagedisplay method executable by a processor. The method comprises:generating one or more of image groups by classifying image data towhich attribution information is added into groups based on theattribute information; and displaying, on a display device, a list of aplurality of thumbnail images of the image data included in each of thegenerated image groups, in a manner in which the plurality of thumbnailimages are piled up for every image group.

It is noted that any combination of the aforementioned components or anymanifestation of the present invention exchanged between methods,devices, systems, computer programs, data structures, recording media,and so forth, is also effective as an embodiment of the presentinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described, by way of example only, withreference to the accompanying drawing, which are meant to be exemplary,not limiting, in which:

FIG. 1 is a view schematically illustrating the internal structure of animage processing apparatus according to an embodiment;

FIG. 2 is a view schematically illustrating the internal structure of animage analyzing unit;

FIG. 3 is a view schematically illustrating the structure of a databasein which the attribute information provided to image data are stored;

FIG. 4 is a view schematically illustrating the internal structure of animage display control unit;

FIG. 5 is a view illustrating an example of a list-screen of image dataoutputted by a list-image display control unit;

FIG. 6 is a view illustrating another example of the list-screen of theimage data outputted by the list-image display control unit;

FIG. 7 is a view illustrating an example of a virtual three-dimensionalspace used in a three-dimensional slide show according to theembodiment;

FIG. 8 is a view illustrating an example of the display screen of thethree-dimensional slide show according to the embodiment;

FIG. 9 is a view illustrating an example of an album editing screenoutputted by an album editing unit;

FIG. 10 is a flowchart illustrating, in a time-series manner, proceduresof processing each unit of an image processing apparatus according tothe embodiment;

FIG. 11 is a flowchart illustrating, in a time-series manner, proceduresof processing slide-show display;

FIG. 12 is a state transition view illustrating transitions of statesthat the image processing apparatus according to the embodiment cantake; and

FIGS. 13A and 13B are views illustrating examples of the relationshipbetween the number of pixels of the original image data and that of thecorresponding thumbnail image when the thumbnail image is generated.

DETAILED DESCRIPTION OF THE INVENTION

The invention will now be described by reference to the preferredembodiments. This does not intend to limit the scope of the presentinvention, but to exemplify the invention.

An image processing apparatus according to an embodiment providesvarious techniques for managing image data, such as a user interface fordisplaying a list of image data stored in a storage device, function ofclassifying images based on the attribute information added to theimages, user interface for helping selection of images, and slide showfor browsing images.

FIG. 1 is a view schematically illustrating the internal structure of animage processing apparatus 100 according to an embodiment. The imageprocessing apparatus 100 according to the embodiment comprises a storageunit 200, an audio processing unit 300, an output unit 400, an imageprocessing unit 500, and a direction receiving unit 800. FIG. 1illustrates the functional configuration for achieving the imageprocessing apparatus 100 according to the embodiment, in which otherconfigurations are omitted. In FIG. 1, each component described as afunctional block for performing various processing can be configuredwith a CPU (Central Processing Unit), a main memory, and other LSIs(Large Scale Integrations) in terms of hardware, and realized with aprogram loaded in the main memory, etc., in terms of software.Accordingly, it will be understood by a person skilled in the art thatthese functional blocks can be realized in various forms by onlyhardware, only software, or combination of hardware and software.Therefore, each functional block should not be limited to any one ofthese.

The storage unit 200 includes a music data storage unit 202 and an imagedata storage unit 204. The music data storage unit stores music datathat is BGM (Back Ground Music) to be reproduced simultaneously with thedisplay of a slide show, which will be described later. The image datastorage unit 204 stores digital image data mainly taken by a user. Thestorage unit 200 can be realizable with a storage device, such as HDD(Hard Disk Drive), SSD (Solid State Drive), or the like, and a removablerecording medium, such as Blu-ray Disc (trademark), etc.

The audio processing unit 300 includes a music data reproducing unit 302and a music data analyzing unit 304. The music data reproducing unit 302reproduces the music data acquired from the music data storage unit 202.The music data analyzing unit 304 analyzes the characteristics of themusic data reproduced by the music data reproducing unit 302. Herein,the “characteristics of the music data” means a characteristic amountextracted from the waveform that the music data uniquely has. Examplesof the characteristics thereof include, for example, music tempo, chordprogression, and frequency distribution, etc. The above analysis can beachieved by using a known technique, such as a twelve-tone analysis (seehttp://www.sony.co.jp/SonyInfo/technology/technology/theme/12tonealalysis_(—)01.html).

The output unit 400 further includes a music data output unit 402 and animage data output unit 404. The music data output unit 402 is aninterface for outputting the music reproduced by the music datareproducing unit 302 to speakers. The image data output unit 404 is aninterface for outputting the image generated by the image processingunit 500 to a display device, such as a television set, monitor, or thelike, and an example of the image data output unit 404 is, for example,an HDMI (High-Definition Multimedia Interface) terminal.

The direction receiving unit 800 acquires a direction for the imageprocessing apparatus 100, which has been issued by a user through a userinterface, such as a non-illustrated mouse, keyboard, game controller,or the like.

The image processing unit 500 performs a series of image processing,such as display of a list of image data, classification of image data,and reproduction of a slide show, based on the direction acquired fromthe user through the direction receiving unit 800. To do that, the imageprocessing unit 500 includes an image classification unit 502, an imageanalyzing unit 600, an image display control unit 700, and an albumediting unit 504.

FIG. 2 is a view schematically illustrating the internal structure ofthe image analyzing unit 600. The image analyzing unit 600 includes ananalysis control unit 602, an aspect ratio acquisition unit 604, anumber-of-pixel acquisition unit 606, and a metadata providing unit 608.

The aspect ratio acquisition unit 604 acquires the aspect ratio of theimage data stored in the image data storage unit 204. Thenumber-of-pixel acquisition unit 606 acquires the total number of pixelsof the image data stored in the image data storage unit 204.Specifically, when it is assumed that the number of pixels in the widthdirection of certain image data is W pixels and that in the heightdirection thereof is H pixels, the aspect ratio acquisition unit 604determines the aspect ratio of the image data to be W:H. In addition,the number-of-pixel acquisition unit 606 determines the total number ofpixels of the image data as W×H pixels.

The metadata providing unit 608 provides, as metadata, attributeinformation to each image data stored in the image data storage unit204, the attribute information indicating the characteristics of eachimage data. The details of the attribute information and a method ofproviding the information will be described later. The analysis controlunit 602 controls the operations of the image analyzing unit 600.Specifically, the analysis control unit 602 acquires a direction fromthe image processing unit 500 to activate the aspect ratio acquisitionunit 604 and the number-of-pixel acquisition unit 606, so that theaspect ratio and the number of pixels of image data are outputted to theimage processing unit 500. In addition, when a new image is added in theimage data storage unit 204, the analysis control unit 602 activates themetadata providing unit 608 to provide attribute information to theadded image.

FIG. 3 is a view schematically illustrating the structure of a databasein which the attribute information provided to image data are stored.The attribute information to be provided to image data is tied with thecorresponding image data and stored in the image data storage unit 204.As illustrated in FIG. 3, the attribute information includes a filename, type, image-taken date and hour, object, album, number of pixels,and display state.

The “type” is the information indicating what type of manner image datahas been generated in. The types of image data include at least an“ordinary image”, “panorama image”, “three-dimensional image”, and“moving image”. The “ordinary image” represents ordinary two-dimensionalimage data, for example, such as JPEG image and BMP image. Although the“panorama image” is also two-dimensional image data, the number ofpixels in the width direction is large relative to that in the heightdirection in comparison with an ordinary image. The “panorama mage” isgenerated by being synthesized from multiple ordinary images taken, forexample, while a viewpoint is continuously being changed.

The “three-dimensional image” is an image including a parallax image forthe left eye and that for the right eye in which an object has beentaken from different viewpoints. Herein, an image of an object in athree-dimensional space, which is viewed from different viewpoints, isreferred to as a “parallax image”. Because the left and right eyes of aperson are spaced apart about 6 cm, parallax is generated between theimage viewed from the left eye and that viewed from the right eye. It issaid that the brain of a person recognizes the depth of an object byusing the parallax images sensed with the left and right eyes.Accordingly, when the parallax image of an object sensed with the lefteye and that sensed with the right eye are projected on the respectiveeyes, the object is recognized as a three-dimensional image having adepth by the person. The “three-dimensional image” can provide athree-dimensional image to a user by being displayed on a dedicateddisplay device, such as a three-dimensional television set.

The “moving image” is a video including images continuous in the timedirection, and examples thereof include, for example, MPEG 4-AVC(Advanced Video Coding) and motion JPEG, etc. Also, a video includingparallax images, such as MPEG 4-MVC (Multiview Video Coding), isincluded in the “moving image”.

The “object” is the information indicating the object taken in imagedata. The “object” is roughly classified into a “person” and objectsother than a person. The “person” is the information on the personconsidered to be most important among the objects and the information ofthe name and birth data of the person taken in image data are stored.The metadata providing unit 608 acquires these information from a userthrough the direction receiving unit 800 to reflect on the database ofattribute information.

Examples of the objects other than a person include, for example, a“flower” “sky”, and “face”, etc. The metadata providing unit 608,including an image recognition engine (not illustrated) for analyzingimage data to recognize the object that has been taken therein,recognizes the object that has been taken to automatically reflect on adatabase of attribute information. The image recognition engine can beachieved by using a known mechanical learning method, for example, suchas SVM (Support Vector Machine) and Boosting, which are learning methodswith teachers. Also, a user can correct the attribute informationautomatically generated by the image recognition engine in the metadataproviding unit 608 by issuing a direction to the metadata providing unit608 through the direction receiving unit 800.

The “album” is the information indicating whether certain image data isincluded in the later-described album. The “display state” is theinformation indicating whether the certain image data is to bedisplayed. When a user sets the attribute of the certain image data to“non-display” through the direction receiving unit 800 and the metadataproviding unit 608, the user cannot browse the certain image data, asfar as the user does not access to the history folder that is adedicated folder for displaying the image data whose display state is“non-display” and that is provided in the image data storage unit 204.By including a “display state” in the attribute information as statedabove, a user can manage images that are not intended to be usually usedfor browsing, etc., but not intended to be deleted from the image datastorage unit 204. The history folder can be treated as something like aso-called “negative film” that is saved without being printed in digitalimage data.

When acquiring a direction from the image display control unit 700, theimage classification unit 502 in FIG. 1 accesses to the database inwhich the aforementioned attribute information are stored, and dividesthe image data into groups by classifying them based on the attributeinformation provided thereto. For example, the image classification unit502 classifies the image data stored in the image data storage unit 204based on the image-taken data and hour thereof. Alternatively, the imageclassification unit 502 can freely divide the image data into groupswith the attribute information being a key, such as classification ofthe image data in which a specific person is taken into one group, anddivision of the image data in which a flower is taken into one group,etc.

Further, the image classification unit 502 may extract a characteristicamount of image data by analyzing it to classify the image data based onthe characteristic amount. Specifically, the image classification unit502 determines the hue distribution of each image data to collect anddivide the image date in which, for example, “red color” is dominantinto one group. Alternatively, the image classification unit 502 maydetermine the luminance distribution of each image data to collect anddivide the image data that are dark as a whole into one group. This isadvantageous in terms of being able to evoke a user's new awareness toimage data by providing the image data classified from variousviewpoints to the user.

Further, the characteristic amount extracted by the image classificationunit 502 can be reflected on the aforementioned database as newattribute information. By accumulating the attribute information fromvarious viewpoints, a group of the image data classified based on thecombinations of the attribute information can provide a further newawareness to a user.

FIG. 4 is a view schematically illustrating the internal structure ofthe image display control unit 700. The image display control unit 700includes an integrated display control unit 702, a slide show displaycontrol unit 704, a single image display control unit 706, and alist-image display control unit 708.

The integrated display control unit 702 controls the operations of theimage display control unit 700 based on the directions from a useracquired trough the direction receiving unit 800. The integrated displaycontrol unit 702 further acquires the image data to be displayed fromthe image data storage unit 204 through the image classification unit502. The integrated display control unit 702 further acquires, from theimage analyzing unit 600, the information on the resolution and aspectratio, etc., of the image to be displayed.

Unless a particular direction from a user is provided, for example,immediately after the activation of the image processing apparatus 100,the integrated display control unit 702 makes the list-image displaycontrol unit 708 display a list of all image data stored in the imagedata storage unit 204.

FIG. 5 is a view illustrating an example of a list-screen 720 of theimage data outputted by the list-image display control unit 708. Thelist-screen 720 generates a belt-shaped region 724 distinguishable froma background image 722 on the background image 722, so that thethumbnail images of the image data stored in the image data storage unit204 are displayed on the belt-shaped region 724.

Herein, the list-image display control unit 708 acquires the aspectratio the image data to be displayed on the list-screen 720 from theaspect ratio acquisition unit 604 through the integrated display controlunit 702, so that the thumbnail image thereof is displayed on thebelt-shaped region 724, while keeping the aspect ratio of the imagedata. A user can freely switches the thumbnail images displayed on thebelt-shaped region 724 from one to another by clicking a non-displayedmouse or controller.

Further, the list-image display control unit 708 acquires the number ofpixels of the image data to be displayed on the list-screen 720 from thenumber-of-pixel acquisition unit 606 through the integrated displaycontrol unit 702, so that the thumbnail image of the image data withmore pixels is displayed, on the belt-shaped region 724, as a largerthumbnail image than that of the image data with less pixels.Specifically, the list-image display control unit 708 acquires the totalnumber of pixels of every image data stored in the image data storageunit 204, so that the image data with the maximum pixels and that withthe minimum pixels are specified in the ordinary images. The list-imagedisplay control unit 708 generates thumbnail images such that the arearatio of the thumbnail image of the image data with the maximum pixelsto that of the image data with the minimum pixels in the ordinary imagesis within a predetermined range.

Herein, the “predetermined range” means a reference range that isdefined to make a display of a list look better by suppressing avariation in the areas of the thumbnail images displayed on thelist-screen 720. Although the reference range may be determined byexperiments, it is determined such that, for example, the area of thethumbnail image of the image data with the maximum pixels is two timesor less larger than that of the image date with the minimum pixels.Thereby, the resolution of the image data has a correlation with themagnitude relationship of the thumbnail images thereof, and accordinglya user can overlook the resolution of every image data stored in theimage data storage unit 204. Also, because the aspect ratio of thethumbnail image displayed on the list-screen 720 is the same as that ofthe image data stored in the image data storage unit 204, the user canalso overlook the outer shape of the image data.

The list-image display control unit 708 does not always display all thethumbnail images such that they are included in the belt-shaped region724. For example, when the ratio of the height relative to the width ofimage data exceeds a predetermined ratio after the aspect ratio thereof,which has been acquired from the aspect ratio acquisition unit 604through the integrated display control unit 702, has been analyzed, thelist-image display control unit 708 displays the thumbnail image of theimage data so as to protrude from the belt-shaped region 724.

Herein, the “predetermined ratio” means a reference ratio defined todisplay, on the list-screen 720, the characteristic of the outer shapeof image data in a more emphasized manner. Although the reference ratiomay be determined by experiments, the reference ratio is applied to, forexample, the case where the ratio of the height to the width of imagedata is greater than three. Because it can be considered that the imagesatisfying the reference ratio is a panorama image, a user can overlookthe types of image data on the list-screen 720.

FIG. 6 is a view illustrating another example of the list-screen of theimage data outputted by the list-image display control unit 708. Theexample illustrated in FIG. 6 illustrates a state in which the imageclassification unit 502 is classifying and dividing into groups theimage data stored in the image data storage unit 204 based on theattribute information thereof. The aforementioned attribute informationcan be arbitrarily used in the division into groups. For example, theimage data can be divided into groups based on the year when the imagedata were taken or on the specific person taken in the image data.

As illustrated in FIG. 6, the image groups generated by the divisioninto groups by the image classification unit 502 are displayed on thebelt-shaped region 724 that is distinguishable from the background, in amanner in which it is represented that the thumbnail images of the imagedata are piled up into bundles of images. The aspect ratio of eachthumbnail image that forms the bundle of images is the same as that ofthe corresponding image data and the magnitude relationship between thethumbnail images have a correlation with that between the numbers ofpixels of the corresponding image data. Thereby, a user can overlook ina group unit the image groups classified based on the attributeinformation. This is advantageous in terms of being able to quicklygrasp the total image of the image data stored in the image data storageunit 204, when it is difficult to overlook the total image thereof onlyby a display of a list of each image because the number of the imagedata stored therein is increased.

When a user selects one of the bundles of images displayed on thebelt-shaped region 724 through the direction receiving unit 800, thelist-image display control unit 708 displays the selected bundle ofimages in a different manner from other bundles of images, in which theselected bundle of images is focused. Specifically, for example, thethumbnail image of the image data included in the selected image groupis displayed on the top surface of the selected bundle of images.Further, the list-image display control unit 708 randomly extracts thepredetermined number of sheets of image data from the selected imagegroup such that an image in which the extracted image data are arrangedis displayed as the background image 722.

Herein, the “predetermined number of sheets” means the reference numberof sheets by which, when a so-called montage image of an image group isgenerated, each thumbnail image has the extent of size in which eachthumbnail image can be recognized. Because the reference number ofsheets is changed in accordance with the size of a display device andthe number of pixels of image data, it may be determined by experiments.A specific example of the “predetermined number of sheets” is, forexample, nine. A user can view part of the image data in the selectedimage group as a sample image, thereby it becomes possible for the userto grasp the reference or theme by which the image group is divided intoa group. It may be made that the “predetermined number of sheets” isfreely set by a user in accordance with his/her preference. In thiscase, the direction receiving unit 800 acquires a direction from theuser to output to the list-image display control unit 708.

Referring back to the descriptions of FIG. 4, the slide show displaycontrol unit 704 further includes a two-dimensional slide show displaycontrol unit 710 and a three-dimensional slide show display control unit712. When a user issues, through the direction receiving unit 800, thedirection that a two-dimensional slide show should be initiated whilethe list-image display control unit 708 is displaying the list-screen720, the two-dimensional slide show display control unit 710 initiates atwo-dimensional slide show. Specifically, the two-dimensional slide showdisplay control unit 710 displays, sequentially one by one, the imagedata included in the image group selected by the user on the displaydevice. When a two-dimensional slide show is initiated while thethumbnail images of the whole image data stored in the image datastorage unit 204, not the image groups thereof, are being displayed onthe list-screen 720, the whole image data stored in the image datastorage unit 204 is considered to be a single image group.

When the two-dimensional slide show display control unit 710 initiates atwo-dimensional slide show, the music data reproducing unit 302reproduces the music data stored in the music data storage unit 202. Themusic data analyzing unit 304 analyzes the characteristic data of themusic reproduced by the music data reproducing unit 302. Thetwo-dimensional slide show display control unit 710 acquires thecharacteristic data of the music analyzed by the music data analyzingunit 304 to control a timing when image data to be displayed is switchedin accordance with the characteristic data of the music. By making atiming when image data is switched be changed with a change in thecharacteristic data of music, it becomes possible to present a differentatmosphere to a user even if the same image group is reproduced by usinga two-dimensional slide show.

Specifically, for example, when the speed of the tempo of music isadopted as the characteristic data of the music, the two-dimensionalslide show display control unit 710 advances a timing when image data isswitched, as the tempo of the music data is faster, and vice versa.Because the correlativity between the tempo of music and the timing whena two-dimensional slide show is switched is increased, it becomespossible for a user to listen to the music suitable for his/her feelingand simultaneously to view image data.

Another example of a two-dimensional slide show will be described. Whena specific person is taken in the image data selected by a user at theinitiation of a two-dimensional slide show, the two-dimensional slideshow display control unit 710 retrieves the database of attributeinformation stored in the image data storage unit 204 to collect theimage data in which the person is taken. The two-dimensional slide showdisplay control unit 710 performs the aforementioned two-dimensionalslide show of the collected image group. In this case, the age of theperson when the picture was taken is displayed as well as the display ofthe image data. This is advantageous in terms of being able to present abrief album regarding the person.

When a user issues, through the direction receiving unit 800, thedirection that a three-dimensional slide show should be initiated afterselecting an arbitrary thumbnail image, while the list-image displaycontrol unit 708 is displaying the list-screen 720 to display thethumbnail images of the whole image data stored in the image datastorage unit 204, the three-dimensional slide show display control unit712 initiates a three-dimensional slide show. Specifically, thethree-dimensional slide show display control unit 712 acquires theattribute information of the image data corresponding to the selectedthumbnail image, and then acquires the predetermined number of sheets ofimage data including the attribute information that are associated withthe acquired attribute information.

Herein, the “attribute information that are associated” means any of theattribute information having an item common in the attribute informationof the image data corresponding to the selected thumbnail image;however, it is preferable that the attribute information can provide anelement of surprise and awareness to a user because the image datahaving such attribute information are not usually edited as an album.Examples of such attribute information include, for example, years whenimage data taken on the same day as the selected image data were taken.Thereby, this is advantageous in terms of being able to create an album,for example, such as “photos on every birthday”.

The three-dimensional slide show display control unit 712 arranges, in ascattered manner, the acquired image data on a virtual three-dimensionalspace set in the non-illustrated work memory. FIG. 7 is a viewillustrating an example of a virtual three-dimensional space 726 used ina three-dimensional slide show according to the embodiment. A camera730, which is a virtual viewpoint moving on a virtually-installed rail728, is present in the virtual three-dimensional space 726. FIG. 7illustrates a state in which image data 732 collectively denoted by areference numeral 732 are arranged in the virtual three-dimensionalspace 726 by the three-dimensional slide show display control unit 712.

The three-dimensional slide show display control unit 712 generates avideo in which the image data 732 are being observed while the camera730 is being moved along the rail 728, i.e., a walk-through video. FIG.8 is a view illustrating an example of the display screen of thethree-dimensional slide show according to the embodiment. As illustratedin FIG. 8, the image data 732 having various sizes are arranged in thevirtual three-dimensional space 726 in various directions and in ascattered manner as if it were a forest of image data. A video generatedby being observed from the viewpoint of the camera 730 moving therein isdisplayed as a three-dimensional slide show.

A user can issue, through the direction receiving unit 800, thedirection that the movement of the camera 730, which is a virtualviewpoint, should be stopped to the three-dimensional slide show displaycontrol unit 712, while a three-dimensional slide show is beingdisplayed. When acquiring the direction that the movement of the camera730 should stopped, the three-dimensional slide show display controlunit 712 stops the movement of the camera 730. Subsequently, thethree-dimensional slide show display control unit 712 displays a videogenerated when the image data that is located, in the virtualthree-dimensional space 726, at the closest position from the stoppedcamera 730 is viewed from the front of the image data.

Herein, the “image data that is located at the closest position from thecamera 730” means the image data in which the distance between thepositional coordinates of the center of the image data and that of thecamera 730 is smallest in the image data arranged in the virtualthree-dimensional space 726. In FIG. 8, because the image data 7321 islocated at the closest position from the stopped camera 730, a videogenerated when the image data 7321 is viewed from the front thereof isto be displayed.

Thereby, it becomes possible to overlook, in a three-dimensional spread,the image data having the aforementioned associated attributeinformation, and accordingly a new awareness to the image data, whichhas been almost forgotten after being stored in the image data storageunit 204, can be presented to the user. The user can carefully observethe image data from the front thereof, the user being interested in theimage data among the image data having the associated attributeinformation.

In the aforementioned descriptions, the case where the camera 730 issolely present as a virtual viewpoint has been described; however, itmay be made that a three-dimensional video including parallax images isgenerated by setting two cameras. This is advantageous in terms of beingable to present a video of the virtual three-dimensional space 726 to auser as a three-dimensional video having a depth by displaying the videoon a display device, such as a three-dimensional television set.

Referring back to the descriptions of FIG. 4, the single image displaycontrol unit 706 further includes a two-dimensional image displaycontrol unit 714, a three-dimensional image display control unit 716,and a moving image display control unit 718.

When a user selects image data through the direction receiving unit 800while the list-image display control unit 708 is displaying thelist-screen 720, the integrated display control unit 702 refers to theattribute information of the image data, and then activates one of thetwo-dimensional image display control unit 714, the three-dimensionalimage display control unit 716 and the moving image display control unit718 based on the type of the image data.

For example, when the image data selected by the user is an “ordinaryimage” or “panorama image”, the integrated display control unit 702activates the two-dimensional image display control unit 714. Thetwo-dimensional image display control unit 714 displays the selectedimage data on the display device. When the selected image data is a“panorama image”, the two-dimensional image display control unit 714displays the whole image by sliding the panorama image data, just likepanning the camera. Alternatively, it may be made that the panoramaimage data is appropriately reduced so as to be displayed at a time onthe whole display device and then displayed. The user can appropriatelyselect, through the direction receiving unit 800, one of the displaymodes in accordance with his/her preference.

When the image data selected by the user is a “three-dimensional image”including a parallax image for the left eye and that for the right eyein which an object has been taken from different viewpoints, theintegrated display control unit 702 activates the three-dimensionalimage display control unit 716. The three-dimensional image displaycontrol unit 716 displays image data in accordance with the type of thedisplay device. The types of the display device include, for example:display devices of the type using a known liquid crystal shutterglasses, and display devices of the type using a Lenticular lens and aParallax Barrier. When the display device does not support athree-dimensional image, the three-dimensional image display controlunit 716 outputs, for example, only a parallax image for the left eye onthe display device.

When the image selected by the user is a “moving image”, the integrateddisplay control unit 702 activates the moving image display control unit718. The moving image display control unit 718 decodes coded movingimages to display on the display device. When the moving image is athree-dimensional video including parallax images, the moving imagedisplay control unit 718 changes an image to be outputted in accordancewith the type of the display device in the same way as thethree-dimensional image display control unit 716.

The functions mainly for viewing the image data stored in the image datastorage unit 204 have been described in the aforementioned descriptions;however, the image processing apparatus 100 according to the embodimentalso provides a user interface for arranging the image data stored inthe image data storage unit 204. Specifically, when a user issues,through the direction receiving unit 800, the direction that editing ofan album should be initiated while the list-image display control unit708 is displaying the list-screen 720, the album editing unit 504 inFIG. 1 displays a user interface for editing an album on the displaydevice.

FIG. 9 is a view illustrating an example of an album editing screenoutputted by the album editing unit 504. The thumbnail images of imagedata are displayed in a list form on the belt-shaped region 724 in thesame way as the list-screen 720. When the user select, through thedirection receiving unit 800, the thumbnail image of image datadisplayed on the belt-shaped region 724, the album editing unit 504simultaneously displays the same image as the thumbnail image of imagedata on a second display region 736 that is different from thebelt-shaped region 724.

When acquiring, from the direction receiving unit 800, a selection of athumbnail image by the user during editing of an album, the list-imagedisplay control unit 708 displays, on the display device, the thumbnailimage of image data selected by the user and that of image data notselected by the user in different modes so as to distinguish one fromanother. In FIG. 9, the thumbnail images of image data selected by theuser are illustrated as thumbnail images in a ghost state that arecollectively denoted by a reference numeral 736.

Herein, the “ghost state” means a state in which the luminance, chroma,or contrast of the thumbnail image is low in comparison with that of thethumbnail image not selected by the user. By displaying the thumbnailimage of image data selected by a user in a ghost state, it becomespossible to distinguish the thumbnail image of image data selected bythe user from the thumbnail image of image data not selected by theuser. Alternatively, it may be made that the thumbnail image of imagedata selected by the user is displayed in an emphasized manner incomparison with the thumbnail image not selected by the user, byincreasing the luminance, chroma, contrast, or the like of the thumbnailimage selected by the user.

Alternatively, the album editing unit 504 may display, during theediting of an album, the thumbnail image of image data selected by auser on the second display region 736 corresponding to the belt-shapedregion 724. For example, when the belt-shaped region 724 and the seconddisplay region 736 form, on the display device, two belt-shaped regionsthat are parallel to each other, the horizontal positional coordinatesof the thumbnail image displayed in the belt-shaped region 724 and thatof the thumbnail image displayed in the second display region 736 aremade to be equal to each other. Thereby, this is advantageous in termsof being able to easily confirm where the image data selected by a useris located in the material images.

As stated above, a user can make a choice of image data by viewing boththe material image data to be included in an album and the image dataalready included in the album. Because the thumbnail images of the imagedata already included in the album can be distinguished among thematerial image data, this is advantageous in terms of being able toeasily make a choice of image data. The case where an album is editedfrom the material images has been described in the aforementioneddescriptions; however, the above user interface can also be used as auser interface for displaying and editing a “trash box” folder, thetrash box being used for arranging unnecessary image data.

FIG. 10 is a flowchart illustrating, in a time-series manner, proceduresof processing each unit of the image processing apparatus 100 accordingto the embodiment. In the flowchart illustrated in FIG. 10, theprocedures of processing each unit will be denoted by combinations of S,which means a step (initial letter of Step), and a number. In addition,when any determination processing is executed in the processing denotedby a combination of S and a number, and when a result of thedetermination is positive, the procedure is denoted by adding Y (initialletter of Yes) to the combination thereof, for example, by (S12/Y).Conversely, the result of the determination is negative, the procedureis denoted by adding N (initial letter of No) thereto, for example, by(S12/N). The processing in the flowchart is initiated when, for example,the image processing apparatus 100 is activated.

When the image processing apparatus 100 is activated, the integrateddisplay control unit 702 makes the list-image display control unit 708display a list of the whole image data stored in the image data storage204 (S10). When the direction that the image data should be classifiedis acquired from a user during the display of a list (S12/Y), the imageclassification unit 502 refers to the database of attribute informationstored in the list-image display control unit 708 based on theclassification key acquired from the user such that the image data isclassified into one or more of image groups (S14).

When the direction that a slide show should be displayed, not thedirection that the image data should be classified (S12/N), is acquiredduring the display of a list (S16/Y), the slide show display controlunit 704 performs a slide show (S18). When the direction that an albumshould be edited, not the direction that a slide show should bedisplayed (S16/N), is acquired (S20/Y), the album editing unit 504displays the user interface for editing an album (S22). When thedirection that an album should be edited is not acquired (S20/N), itadvances to the next step S24.

Before the direction that the image processing apparatus 100 should beterminated is acquired from a user through the direction receiving unit800 (S24/N), the image processing apparatus 100 continues theaforementioned processing steps. When the direction that the imageprocessing apparatus 100 should be terminated is acquired from a userthrough the direction receiving unit 800 (S24/Y), the processing isended.

FIG. 11 is a flowchart illustrating, in a time-series manner, theprocedures of processing a slide show display in the slide show displaycontrol unit 704, which illustrates the details of S18 in FIG. 10. Theintegrated display control unit 702 acquires, through the directionreceiving unit 800, the type of the slide show to be performed from auser (S26). Herein, the “type of the slide show” means a two-dimensionalslide show or a three-dimensional slide show.

When the direction that a two-dimensional slide show should be performedis acquired from a user (S28/Y), the two-dimensional slide show displaycontrol unit 710 is activated by the integrated display control unit 702to acquire the result of analyzing music data from the music dataanalyzing unit 304 (S30). Subsequently, the two-dimensional slide showdisplay control unit 710 performs a two-dimensional slide showsimultaneously with the reproduction of music data by the music datareproducing unit 302 (S32).

When the direction that a three-dimensional slide show should beperformed is acquired from a user (S28/N), the three-dimensional slideshow display control unit 712 is activated by the integrated displaycontrol unit 702 to perform a three-dimensional slide show (S34). Whenthe slide show performed by the two-dimensional slide show displaycontrol unit 710 or the three-dimensional slide show display controlunit 712 is terminated, the processing is ended.

The operations with the aforementioned configuration will be describedwith reference to FIG. 12.

FIG. 12 is a state transition view illustrating transitions of statesthat the image processing apparatus 100 according to the embodiment cantake. In the state transition view illustrated in FIG. 12, the statesthat the image processing apparatus 100 can take will be denoted bycombinations of ST, which means a state (abbreviation of STatus), and anumber.

When a user activates the image processing apparatus 100 according tothe embodiment, the integrated display control unit 702 activates thelist-image display control unit 708 to make the image processingapparatus 100 be in a display of a list state ST10. In the display of alist state ST10, a user interface as illustrated in FIG. 5 is displayedon the display device. The user can have the image processing apparatus100 make a transition to the below-described five states by operating acontroller, etc., with the display of a list state ST10 being a startingpoint. That is, the five states include an image classification stateST12, a two-dimensional slide show state ST14, a three-dimensional slideshow state ST16, an album editing state ST18, and a trash box displaystate ST20.

When a user inputs a classification key by operating a controller, etc.,in the image classification state ST12, the image classification unit502 classifies the image data recorded in the image data storage unit204. When the image classification unit 502 classifies the image data,the image classification state ST12 automatically makes a transition tothe display of a list state ST10. Herein, when the image classificationstate ST12 makes a transition to the display of a list state ST10, astate in which the image data recorded in the image data storage unit204 are classified into groups based on the attribute informationthereof, is displayed on the display device as illustrated in FIG. 6. Ofcourse, when a user issues the direction that the thumbnail images ofthe all image data should be displayed in a list form, instead of theclassification of the image data, in the image classification stateST12, the user interface as illustrated in FIG. 5 is displayed.

When the user issues the direction that a two-dimensional slide showshould be performed in the display of a list state ST10, thetwo-dimensional slide show display control unit 710 performs atwo-dimensional slide show, and the image processing apparatus 100 makesa transition to the two-dimensional slide show state ST14. When thetwo-dimensional slide show is terminated, the image processing apparatus100 automatically returns to the display of a list state ST10. Likewise,when the user issues the direction that a three-dimensional slide showshould be performed in the display of a list state ST10, thethree-dimensional slide show display control unit 712 performs athree-dimensional slide show, and the image processing apparatus 100makes a transition to the three-dimensional slide show state ST16. Whenthe three-dimensional slide show is terminated, the image processingapparatus 100 automatically returns to the display of a list state ST10.

Herein, a restriction may be added in which the display of a list stateST10 can make a transition to the three-dimensional slide show stateST16 only in the state where, in the display of a list state ST10, thethumbnail images of all the image data are displayed in a list forminstead of the classification of the image data. It is because a newawareness is more likely to occur by selecting the image data to bedisplayed in a three-dimensional slide show from a population as largeas possible, since a three-dimensional slide show is a user interfaceincluding the viewpoint of providing a new awareness to a user as statedabove.

When a user issues the direction that an album editing should beperformed in the display of a list state ST10, the album editing unit504 displays a user interface for editing an album on the displaydevice, and the image processing apparatus 100 makes a transition to thealbum editing state ST18. When the album editing is terminated, theimage processing apparatus 100 automatically returns to the display of alist state ST10. Likewise, when a user issues, in the display of a liststate ST10, the direction that the trash box should be edited, anon-illustrated trash box editing unit displays, on the display device,a similar user interface as that for album editing, and the imageprocessing apparatus 100 makes a transition to the trash box displaystate ST20. When the trash box editing is terminated, the imageprocessing apparatus 100 automatically returns to the display of a liststate ST10.

As stated above, by making it possible that, to the states other thanthe display of a list state ST10 (the image classification state ST12,the two-dimensional slide show state ST14, the three-dimensional slideshow state ST16, the album editing state ST18, and the trash box displaystate ST20), only the display of a list state ST 10 makes a transition,it becomes possible to provide a unified user interface to a user inwhich the display of a list state ST10 is a starting point.

According to the embodiment, a technique in which a large amount ofdigital image data can be readily managed can be provided as statedabove.

The present invention has been described above based on the preferredembodiment. The aforementioned embodiment has been described forexemplary purpose only, and is by no means intended to be interpretedrestrictively. Rather, it can be readily understood by a person skilledin the art that various modifications may be made by making variouscombinations of the aforementioned components or processes, which arealso encompassed in the scope of the present invention.

In the aforementioned descriptions, the case where the list-imagedisplay control unit 708 displays the thumbnail images of the image datastored in the image data storage unit 204 on the belt-shaped region 724,which can be distinguished from the background image 722, has beendescribed; however, the region in which the thumbnail images aredisplayed may have a shape other than a belt shape. For example, thelist-image display control unit 708 displays thumbnail images in thewhole display region so as to wholly cover the background. In this case,the number of the thumbnail images presented to a user can be increasedin comparison with the case where thumbnail images are displayed in thebelt-shaped region 724, and hence this is advantageous when a userintends to quickly overlook the whole image data stored in the imagedata storage unit 204.

The case where the music data reproducing unit 302 reproduces music datasimultaneously with the performance of a two-dimensional slide show bythe two-dimensional slide show display control unit 710 has beendescribed in the above descriptions; however, the music data reproducingunit 302 may reproduce music data when the three-dimensional slide showdisplay control unit 712 performs a three-dimensional slide show. Inthis case, the three-dimensional slide show display control unit 712acquires, from the music data analyzing unit 304, the result ofanalyzing the music data to changes a mode of performing athree-dimensional slide show in accordance with the result thereof. Forexample, as the tempo of music data is faster, the moving speed of thecamera 730 that moves within the virtual three-dimensional space 726 isincreased. This is advantageous in terms of being able to listen to themusic suitable for the feeling of a user and simultaneously to view athree-dimensional slide show, because the correlativity between thetempo of music and the rate of progression of the three-dimensionalslide show is increased.

The case where the image classification unit 502 classifies image databased on the attribute information provided to the image data has beendescribed in the above descriptions; however, the image classificationunit 502 may classify the image data to which no attribute informationis provided as one group. In this case, when the image data to which noattribute information is provided is newly stored in the image datastorage unit 204, the image classification unit 502 collectivelypresents these image data to a user. This is advantageous to the user interms of being able to manually edit attribute information.

The fact that, of the attribute information provided to image data, whenthe “display state” is the “non-display”, the image data can be viewedonly in the history folder, has been described in the abovedescriptions; however, the image classification unit 502 may storeparticular images in the image data storage unit 204 and simultaneouslyprovide the attribute of “non-display” to the particular imagesautomatically. The “particular images” means, for example, either ofJPEG images corresponding to a three-dimensional image and JPEG imagescorresponding to a Raw image. This is advantageous in terms of beingable to prevent the redundancy due to the display of a list of similarimage data.

The case where the list-image display control unit 708 displays, on thebelt-shaped region 724, the thumbnail image of the image data with morepixels as a larger thumbnail image than that of the image data with lesspixels has been described in the above descriptions. Such a thumbnailimage is generated by the list-image display control unit 708 reducingthe image data; however, the list-image display control unit 708 maychange the enlargement/reduction ratio in accordance with the number ofpixels of the original image data. Herein, the “enlargement/reductionratio” means the ratio of the number of pixels in the short side of thethumbnail image to that of the original image data when the thumbnailimage is generated by reducing the image data while the aspect ratio ofthe image data is being kept. As the enlargement/reduction ratio issmaller, the generated thumbnail image becomes smaller.

FIG. 13A is a view illustrating an example of the relationship betweenthe number of pixels of the original image data and that of thecorresponding thumbnail image when the thumbnail image is generated.FIG. 13A illustrates the fact that the variation in the number of pixelsof the thumbnail image is smaller than that of the original image data,as a result of reducing the enlargement/reduction ratio as the number ofpixels of the original image data is larger. Further, FIG. 13Aillustrates that the minimum of the number of pixels of the generatedthumbnail image is q0 pixels.

FIG. 13B is a view illustrating another example of the relationshipbetween the number of pixels of the original image data and that of thecorresponding thumbnail image when the thumbnail image is generated. Inthe same way as the example of FIG. 13A, the minimum of the number ofpixels of the generated thumbnail image is q0 pixels. In the graphillustrated in FIG. 13B, there exist inflection points when the numberof pixels of the original image is p0, p1 and p2. When the number ofpixels of the original image is smaller than or equal to p0, theincrease in the enlargement/reduction ratio is increased as the numberof pixels thereof becomes smaller, thereby the change in the number ofpixels of the thumbnail image becomes moderate. When the number ofpixels of the original image is between p0 and p1, the relationshipbetween the two is the same as the case illustrated in FIG. 13A.

When the number of pixels of the original image is between p1 and p2,the decrease in the enlargement/reduction ratio is increased as thenumber of pixels thereof becomes larger, thereby the change in thenumber of pixels of the generated thumbnail image becomes moderate. Whenthe number of pixels of the original image is larger than p2, thelist-image display control unit 708 determines the enlargement/reductionratio such that the number of pixels of the thumbnail image becomes aconstant value of q1 pixels.

In each of the examples illustrated in FIGS. 13A and 13B, the variationin the number of pixels of the thumbnail image is smaller than that ofthe original image data. As a result, well-uniformed display can beachieved when the thumbnail images are displayed in a list form whilethe magnitude relationship of the numbers of pixels of the originalimage is being kept. Thereby, it can be prevented that, when there is agreat difference between the numbers of pixels of the original images tobe displayed, it becomes difficult to view the thumbnail image with lesspixels because it is flattened.

1. An image display apparatus comprising: a number-of-pixel acquisitionunit configured to acquire the number of pixels of image data stored ina storage device; an aspect ratio acquisition unit configured to acquirethe aspect ratio of the image data; and a display control unitconfigured to display a list of a plurality of thumbnail images of theimage data on a display device, wherein the display control unitdisplays a thumbnail image of image data having more pixels as a largerthumbnail image than that of image data having less pixels, whilekeeping the aspect ratio of the image data.
 2. The image displayapparatus according to claim 1, wherein the display control unitacquires and analyzes the number of pixels of the image data stored inthe storage device from the number-of-pixel acquisition unit, so that asize of a thumbnail image of the image data is determined based on thetotal number of pixels of an image having the maximum number of pixelsor of an image having the minimum number of pixels.
 3. The image displayapparatus according to claim 1, wherein the display control unitdisplays a thumbnail image of image data as a belt-shaped regiondistinguishable from a background of an image, and displays one of theplurality of thumbnail images of the image data to be displayed so as toprotrude from the belt-shaped region.
 4. An image display methodexecutable by a processor comprising: acquiring the number of pixels ofimage data stored in a storage device; acquiring the aspect ratio of theimage data; and displaying, on a display device, a list of a pluralityof thumbnail images of the image data having more pixels as a largerthumbnail image than that of the image data having less pixels, whilekeeping the aspect ratio of the image data.
 5. A program productexecutable by a computer, comprising: a module configured to acquire thenumber of pixels of image data stored in a storage device; a moduleconfigured to acquire an aspect ratio of the image data; and a moduleconfigured to display, on a display device, a list of a plurality ofthumbnail images of the image data with image data having more pixels asa larger thumbnail image than that of the image data having less pixels,while keeping the aspect ratio of the image data.
 6. An image displayapparatus comprising: a number-of-pixel acquisition unit configured toacquire the number of pixels of image data stored in a storage device;and an image display control unit configured to generate a plurality ofthumbnail images in a manner in which an enlargement/reduction ratioused when a thumbnail image of image data with more pixels is to begenerated is made smaller than that used when the thumbnail image of theimage data with less pixels is to be generated, and to display a list ofthe generated thumbnail images on a display device.